Generalized supervised meta-blocking

Entity Resolution is a core data integration task that relies on Blocking to scale large datasets. Schema-agnostic blocking achieves very high recall, requires no domain knowledge and applies of any structuredness schema heterogeneity. This comes at the cost many irrelevant candidate pairs (i.e., comparisons), which can be significantly reduced by Meta-blocking techniques leverage entity co-occurrence patterns inside blocks: first, entities are weighted in proportion their matching likelihood, then, pruning discards with lowest scores. Supervised goes beyond this approach combining multiple scores per comparison into feature vector fed binary classifier. By using probabilistic classifiers, Generalized associates every pair candidates score used algorithm. For higher effectiveness, new weighting schemes examined as features. Through extensive experiments, we identify best algorithms, optimal sets features, well minimum possible size training set.